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1.0  EXECUTIVE  SUMMARY 


In  September  2008,  Battelle  completed  construction  of  a  new  airborne,  time-domain 
electromagnetic  system  for  mapping  and  detecting  unexploded  ordnance  (UXO).  This  system 
was  developed  with  corporate  funds  on  the  basis  of  successful  evaluation  of  a  prototype  system 
under  Environmental  Security  Technology  Certification  Program  (ESTCP)  Project  200101.  This 
system  has  been  developed  to  address  shortcomings  of  magnetometer-based  systems  where  the 
presence  of  basalt  flows  or  other  iron-bearing  soils  and  rocks  impede  the  perfonnance  of 
magnetometer  systems.  Although  this  is  not  a  universal  problem,  it  occurs  with  varying  degrees 
of  severity  at  many  sites,  in  particular,  in  the  western  continental  United  States  as  well  as 
portions  of  Hawaii  and  Alaska. 

The  Battelle  Time-Domain  Electromagnetic  (TEM)-8  system  is  contained  within  a  12  by  3 
meters  (m)  rectangular  boom  structure  with  a  two-lobed  transmitter  loop  composed  of  two  3  by  4 
m  rectangles.  There  are  four  receivers  on  each  side  of  the  aircraft,  located  within  4  m  tube 
segments  that  are  oriented  parallel  to  the  long  axis  of  the  boom  structure.  As  with  most  transient 
electromagnetic  (EM)  systems,  a  current  is  established  in  the  transmitter  loop,  then  rapidly 
switched  off,  inducing  a  secondary  magnetic  field  in  the  earth.  The  decay  of  the  secondary 
magnetic  field  is  measured  in  the  receiver  coils.  The  central  third  of  the  boom  structure  is 
directly  under  the  helicopter  and  is  inactive  requiring  interleave  flight  lines  to  achieve  full 
coverage  of  the  underlying  subsurface. 

Two  sites  near  Albuquerque,  NM,  were  selected  for  a  February  2009  demonstration — a  617-acre 
portion  of  the  Fonner  Kirtland  Precision  Bombing  Range  (FKPBR),  and  the  Kirtland  Precision 
Bombing  Range  (PBR)-S12  Target  (S12).  The  FKPBR  area  was  chosen  to  enable  comparison 
with  previous  ESTCP  demonstrations  of  wide  area  assessment  (WAA)  technologies  and  because 
moderate  basaltic  interference  has  been  recognized  in  previous  WAA  demonstrations.  A  100- 
acre  area  within  the  FKPBR  area  was  specified  for  emplacement  of  seed  items.  The  seed  items 
were  emplaced  under  the  direction  of  the  ESTCP  Program  Office  without  involvement  from 
AMEC  or  Battelle.  Validation  of  data  from  this  area  was  made  by  the  Institute  for  Defense 
Analyses  (IDA)  by  comparing  dig  lists  to  a  master  list  of  seeded  items.  A  total  of  1 10  seed  items 
were  emplaced,  including  8 1 -millimeter  (mm)  and  4.2-inch  mortars,  105  mm  projectiles  and 
high  explosive  anti-tank  warhead  (HEAT)  rounds,  and  155  m  projectiles.  IDA  determined  that 
TEM-8  detected  109  of  the  110  seed  items,  missing  one  4.2-inch  mortar  by  only  1  centimeter 
(cm)  outside  the  150  cm  halo.  The  mean  miss  distance  was  0.34  m  with  a  standard  deviation  of 
0.23  m. 

The  PBR-S12  site  was  chosen  as  being  representative  of  sites  where  ground-based  and  airborne 
magnetometer  data  are  ineffective  for  UXO  mapping  and  detection  due  to  interference  from  a 
basalt  flow  that  is  exposed  across  the  entire  site.  Airborne  magnetometer  data  that  were  acquired 
in  2002  with  the  Oak  Ridge  Airborne  Geophysical  System  (ORAGS)-Arrowhead  system  at  PBR- 
S12  indicated  no  distinguishable  response  to  ordnance  at  the  target,  even  though  a  concentration 
of  M3 8  scrap  was  observed  at  the  surface  near  the  center  of  the  target.  The  ordnance  at  the  site 
consists  almost  exclusively  of  M3 8  fragments,  with  occasional  M3 8s  that  are  largely  intact. 
From  the  airborne  data,  two  100  by  100  m  grids  were  selected  for  validation.  Ground-based 
geophysical  data  were  collected  in  these  grids  with  a  Geonics  EM61  MK2  (EM61)  time  domain 
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metal  detection  system.  A  total  of  327  anomalies  detected  by  the  TEM-8  and  EM61  were 
evaluated  by  excavation.  The  validation  indicated  that  all  but  two  of  the  excavated  items  over 
5  lb  were  detected  by  TEM-8,  and  78%  of  the  excavated  items  weighing  between  1  and  5  lb  were 
detected.  The  TEM-8  detected  3 1%  of  the  M38  fragments  that  weighed  less  than  1  lb. 

The  TEM-8  demonstration  exceeded  all  perfonnance  objectives  established  in  advance  of  the 
test.  The  results  indicate  that  TEM-8  fills  an  important  niche  in  WAA  assessments  by  enabling 
the  use  of  lower-cost  airborne  detection  systems  in  areas  where  moderate  to  severe  basaltic 
interference  causes  magnetometer  systems  to  encounter  many  false  positive  anomalies  or  to  miss 
ordnance  altogether.  The  TEM-8  may  also  prove  beneficial  as  a  primary  or  supplemental  system 
in  areas  where  magnetometer  system  performance  is  acceptable. 
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2.0  INTRODUCTION 


2.1  BACKGROUND 

It  is  estimated  that  UXO  may  contaminate  15  million  acres  or  more  within  the  United  States 
alone.  A  need  for  improved  technologies  for  mapping  and  detecting  UXO  has  led  to 
development  of  a  sequence  of  airborne  reconnaissance  systems,  using  electromagnetic  (Beard  et 
ah,  2004;  Doll  et  ah,  2003;  Holladay  et  ah,  2006)  and  magnetic  (Gamey  et  ah,  2004  )  sensors. 

2.2  OBJECTIVES  OF  THE  DEMONSTRATION 

There  are  two  distinct  objectives  for  this  demonstration.  First  and  foremost,  the  demonstration 
provides  a  means  of  assessing  the  effectiveness  of  the  new  TEM-8  airborne  TEM  system  and 
comparing  this  technology  with  airborne  magnetometer  systems  for  mapping  and  detecting 
ordnance.  The  assessment  is  performed  from  data  collected  in  two  distinct  geologic  areas  in 
New  Mexico.  One  area  contains  mild  geologic  interference,  and  the  other  contains  basaltic 
terrain  that  poses  severe  geologic  interference  associated  with  the  magnetite  content  of  the  basalt. 

A  second  objective  of  the  demonstration  is  to  assess  the  effectiveness  of  the  TEM-8  system  for 
WAA  applications.  The  Demonstration  Site  for  this  project  has  been  used  for  previous  WAA 
demonstrations  and  therefore  allows  a  basis  for  achieving  this  second  objective. 

2.3  REGULATORY  DRIVERS 

No  specific  regulatory  drivers  influenced  this  technology  demonstration.  UXO-related  activity  is 
generally  conducted  under  Comprehensive  Environmental  Response,  Compensation,  and 
Liability  Act  (CERCLA)  authority.  Regardless  of  a  lack  of  specific  regulatory  drivers,  many 
Department  of  Defense  (DoD)  sites  and  installations  are  aggressively  pursuing  innovative 
technologies  to  address  a  variety  of  issues  associated  with  ordnance  and  ordnance-related 
artifacts  (e.g.  burial  sites)  that  resulted  from  weapons  testing  or  training  activities.  These  issues 
include  munitions  footprint  reduction  and  site  characterization  that  are  areas  of  particular  focus 
for  this  technology  demonstration.  In  many  cases,  the  prevailing  concerns  at  military  munitions 
sites  can  lead  to  airborne  surveying  to  assess  the  extent  of  munitions  impacted  areas  and 
remediation  efforts  despite  the  absence  of  relevant  regulatory  drivers  and  mandates. 
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3.0  TECHNOLOGY 


3.1  TECHNOLOGY  DESCRIPTION 

The  Battelle  TEM-8  system  is  a  helicopter-mounted  time-domain  electromagnetic  metal 
detection  system  that  was  designed  for  mapping  UXO  in  areas  where  geologic  conditions  cause 
magnetometer-based  systems  to  be  ineffective.  Figure  1  depicts  the  system  during  aerial 
maneuvers.  First  and  foremost,  it  has  been  anticipated  that  the  TEM-8  system  will  prove 
superior  to  magnetometer  systems  where  the  presence  of  basalt  flows  impedes  the  perfonnance 
of  the  magnetometer  systems.  Although  this  is  not  a  universal  problem,  it  occurs  with  varying 
degrees  of  severity  at  many  sites  throughout  the  United  States  and,  in  particular,  in  the  western 
continental  United  States  as  well  as  portions  of  Hawaii  and  Alaska.  Secondarily,  an 
electromagnetic  system  would  prove  beneficial  at  sites  where  nonferrous  ordnance  might  occur, 
or  where  more  attributes  derivable  from  TEM-8  data  could  provide  a  cost-effective  reduction  in 
the  number  of  targets  requiring  further  ground-based  evaluation. 


Figure  1.  TEM-8  airborne  electromagnetic  system. 

In  2002,  under  ESTCP  Project  200101,  Battelle  staff  (then  at  Oak  Ridge  National  Laboratory 
[ORNL])  evaluated  a  prototype  TEM  system  for  mapping  and  detecting  UXO  (ORNL,  2004a). 
This  study  demonstrated  excellent  sensitivity  to  ordnance  when  the  system  (ORAGS-TEM)  was 
flown  at  sufficiently  low  altitudes.  The  system,  however,  lacked  the  necessary  efficiency  for 
production-scale  operations  as  it  contained  only  two  receiver  channels. 

Battelle  committed  corporate  funds  to  design  and  construct  a  new  TEM  system  as  a  result  of 
successful  testing  conducted  in  2002.  This  system  is  similar  to  the  ORAGS-TEM  system  in 
many  regards  but  contains  eight  receiver  coils  instead  of  two.  The  Battelle  TEM-8  system  is 
housed  within  a  3  m  by  12  m  rectangular  boom  structure  with  a  two-lobed  transmitter  loop 
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composed  of  two  3  m  by  4  m  rectangles.  There  are  four  receivers  on  each  side  of  the  aircraft, 
located  within  4  m  tube  segments  that  are  oriented  parallel  to  the  long  axis  of  the  boom  structure 
(as  presented  in  Figure  1).  As  with  other  transient  EM  systems,  a  current  is  established  in  the 
transmitter  loop.  This  current  is  then  rapidly  switched  off,  inducing  a  secondary  magnetic  field 
in  the  earth  and  any  metallic  objects  present  including  target  ordnance.  The  decay  of  this 
secondary  magnetic  field  is  measured  in  the  receiver  coils  as  a  voltage  response.  The  central 
third  of  the  boom  structure  directly  under  the  helicopter  is  inactive,  thus  making  it  is  necessary  to 
interleave  flight  lines  in  order  to  achieve  full  coverage  of  the  underlying  subsurface.  The  Battelle 
TEM-8  system  was  first  deployed  in  a  shakedown  test  at  Battelle’s  West  Jefferson  Ohio  UXO 
Airborne  System  Test  Site  in  November  2007  as  part  of  this  ESTCP  project.  The  TEM-8  system 
was  subsequently  deployed  at  Marine  Corps  Air  Ground  Combat  Center  (MCAGCC) 
Twentynine  Palms,  CA,  and  at  Royal  Australian  Air  Force  (RAAF)  Amberley,  Australia,  during 
calendar  year  2008. 

3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

Airborne  surveys  for  UXO  are  capable  of  providing  data  for  characterizing  potential  UXO 
contamination  at  a  site  with  a  considerably  lower  cost  per  acre  than  ground-based  systems. 
Furthennore,  the  airborne  data  is  acquired  in  a  shorter  period  of  time.  Airborne  systems  are 
particularly  effective  at  sites  having  low-growth  vegetation  and  minimal  topographic  relief. 
They  can  also  be  used  where  heavy  brush,  wetland,  mud,  or  rocky  terrain  makes  it  difficult  to 
conduct  ground-based  surveys.  Sensitivity  of  airborne  systems  is  clearly  lower  than  that  of 
ground-based  systems  (e.g.,  towed  array  surveys  using  the  Vehicular  Simultaneous 
Electromagnetic  Induction  and  Magnetometer  System  [VSEMS]),  which  can  operate  with 
sensors  at  less  than  0.5  m  above  ground  level  (AGL). 

Airborne  and  ground  magnetometer  systems  are  susceptible  to  interference  from  magnetic  rocks 
and  magnetic  soils.  Rugged  topography  or  tall  vegetation  can  limit  the  utility  of  helicopter 
systems,  necessitating  survey  heights  that  are  too  high  for  resolution  of  individual  UXO  items. 
The  performance  of  ground-based  electromagnetic  systems  has  been  proven  superior  to 
magnetometer  systems  where  magnetite  bearing  basaltic  rocks  are  problematic.  The  airborne 
TEM-8  system  demonstrates  a  similar  advantage  over  airborne  magnetometer  systems  under  the 
same  geologic  conditions.  Where  magnetic  geology  is  not  problematic,  TEM-8  does  not  perfonn 
as  well  as  magnetometer  systems  at  altitudes  2  m  and  higher. 

The  primary  advantage  of  the  airborne  technology  is  the  capability  to  survey  large  areas  more 
quickly  and  less  expensively  than  conventional  ground-based  surveys.  Ground-based  EM 
systems  are  preferred  over  ground-based  magnetometer  systems,  even  when  geology  is  not 
magnetic  when  the  target  ordnance  are  at  shallow  depths.  Ground-based  magnetometer  systems 
are  usually  preferred  over  ground-based  EM  systems  for  deep  ordnance  items.  Similarly, 
airborne  EM  systems  are  less  sensitive  to  deep  ordnance  than  airborne  magnetometer  systems. 
The  primary  advantage  of  airborne  EM  systems  over  airborne  magnetometer  systems  is  the 
ability  to  collect  meaningful  data  within  sites  containing  significant  magnetic  geologic 
interference,  or  where  UXO  targets  may  be  largely  non-magnetic.  Airborne  systems  also  have 
an  advantage  in  areas  where  ground  access  is  limited  or  difficult  due  to  surface  conditions 
(swamp  or  marsh)  or  inherent  danger  (exposure  to  UXO  or  other  contaminants).  Areas  with  a 
sensitive  ecological  environment  may  also  benefit  from  the  less  intrusive  airborne  technologies. 
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4.0  PERFORMANCE  OBJECTIVES 


Demonstration  performance  is  generally  assessed  by  comparison  of  the  processed/analyzed 
results  from  the  demonstration  survey  with  established  baselines  from  previous  assessments  or 
demonstrations.  In  this  case,  however,  there  are  no  previous  airborne  EM  surveys  that  can  be 
used  in  comparison  (other  than  the  informal  results  from  the  ORAGS-TEM  system  at  the 
Badlands  Bombing  Range  [ORNL,  2004a]),  so  the  assessment  must  be  done  by  comparison  with 
airborne  magnetometer  results,  ground-based  EM  results,  or  projected  performance  metrics. 
Evaluation  of  seeded  items  provides  a  basis  for  assessing  the  detection  of  ordnance  items. 
Demonstration  success  is  defined  as  the  successful  acquisition  of  airborne  geophysical  data 
(without  any  aviation  incident  or  airborne  system  failure)  and  meeting  the  baseline  requirements 
for  system  performance  as  established  and  presented  in  Table  1.  Several  factors  associated  with 
data  acquisition  cannot  be  strictly  controlled,  such  as  aircraft  altitude  and  attitude.  Therefore,  it 
is  difficult  to  obtain  as  much  precision  and  repeatability  as  one  might  want  for  comparing  system 
performance.  Depth  estimates  are  not  made  from  the  TEM-8  data,  but  the  depths  of  emplaced 
and  excavated  targets  will  be  recorded  as  a  parameter  for  characterizing  the  depth  sensitivity  of 
the  system.  Items  that  are  detected  as  well  as  those  that  are  missed  have  been  reviewed  to  assess 
any  role  that  their  depth  may  have  played.  Data  were  acquired  at  two  sites,  FKPBR  and  PBR- 
S12,  described  in  greater  detail  in  Section  5.  Figure  2  depicts  the  two  areas  adjacent  to  Double 
Eagle  Airport  that  were  surveyed  in  previous  ESTCP  WAA  projects.  Figure  3  shows  total  field 
magnetic  anomaly  map  of  the  area  at  FKPBR  that  was  surveyed  by  Sky  Research  in  an  earlier 
2005-6  ESTCP  WAA  project.  Both  figures  are  discussed  in  more  detail  in  Section  5. 

Table  1.  Performance  objectives  of  TEM-8  system. 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Results 

Quantitative  Performance  Objectives 

Detection1  of  all 
munitions  of  interest 
(moderate  geologic 
interference) 

Percent  detected  of 
seeded  items 
(FKPBR) 

Location  of  seeded 
items 

Prioritized  dig  list 

Probability  of  detection 
(Pd)  per  item  as 
summarized  in  Table  2 

Detected  99.1%  of 
seeded  items  at 

FKPBR,  exceeding 
success  criteria  for  all 
ordnance  types  per 

Table  2 

Detection1  of  all 
munitions  of  interest 
(extreme  geologic 
interference) 

Percent  detected  of 
items  identified  in 
EM61  survey 
(PBR-S12) 

Dig  list  from  EM6 1 
survey 

Prioritized  dig  list 
from  TEM-8 

Locations  from 
validation  survey 

Detect  90%  of  M38  at 
PBR-S12 

Detected  95%  of 
largely  intact  M3  8s  at 
PBR-S12 

Reduction  of  false 
alanns  relative  to 
magnetometer 
measurements 
(PBR-S12) 

Percent  of  False 
positives2  at 

PBR-S12 

Dig  list  from  EM6 1 
survey 

Prioritized  dig  list 
from  TEM-8 

Locations  from 
validation  survey 

Equal  numbers  of  false 
positives  and  successful 
detections  at  SNR  of  2: 1 
for  M3  8s  or  other 
ordnance  types  at  PBR- 
S12 

81.3%  of  digs  from 
TEM-8  list  were 
successful  in  finding 
M38s  or  M38  frag 
(157  of  193  digs). 

False  positives 
constituted  18.7% 
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Table  1.  Performance  objectives  of  TEM-8  system  (continued), 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Results 

Improved  signal-to- 
noise  ratio  (SNR)  of 
EMI  over  magnetics  in 
adverse  geologic 
settings  (PBR-S12) 

SNR3  at  PBR-S12. 

Amplitude  of  TEM-8 
detections 

Amplitudes  of 
previous  magnetic 
detections 

Gridded  TEM  and 
magnetic  data  sets 

TEM  SNR  >  mag  SNR 

At  PBR-S12,  the 

TEM-8  SNR  is 
estimated  to  be  more 
than  1200  times  as 
large  as  the 
magnetometer  SNR. 

At  FKPBR,  where 
geology  is  less  severe, 
a  representative  quiet 
area  had  similar  SNRs, 
while  in  a  noisier 
background;  the  TEM- 
8  SNR  was  about  four 
times  that  of  the 
magnetometer  system. 

Location  accuracy 

Average  error  and 
standard  deviation 
for  seed  (FKPBR) 
and  excavated 
(PBR-S12)  items 

Location  of  seed 
items  surveyed  to 
accuracy  of  1  cm 
(FKPBR) 

Locations  of 
excavated  items  to 
accuracy  of  2  cm 
(PBR-S12) 

Estimated  locations 
from  analysis  of 
TEM-8  data 

90%  of  estimated 
locations  for  detected 
ordnance  within  1.50  m 
of  actual;  AN  and  AE 
<0.5  m  gN  and 
gE  <0.5  m 

99%  were  within  1.5  m 
at  FKPBR,  with  mean 
miss  distance  of  0.34, 
and  s.d.  of  0.23.  At 
PBR-S12,  all  but  three 
of  157  detected  items 
were  within  1.5  m,  and 
the  mean  miss  distance 
was  0.58  m  with 

0.31  m  s.d. 

Production  rate 

Number  of  acres  of 
data  collection  per 
day 

Log  of  field  work 

Survey:  125  acres  per  day 

22.35  acres/hr  or 

106.2  acres/day 
Afternoon  winds 
caused  short  days 

Qualitative  Performance  Objectives 

Ease  of  use 

Pilot  approval 

Flight  performance  is 
acceptable. 

Terrain/vegetation 

restrictions 

Acceptable  for  targets 
of  interest 

Sensitivity  falls  off 
markedly  above  about 

3. 5-4.0  m  altitude, 
where  higher  altitude 
was  caused  by 
vegetation. 

Aerodynamic  stability 

Safety,  certification, 
no  restrictions 

Airspeed  limited  to  70 
knots 

Detection  capabilities 

Superior  delineation 
of  ordnance 
compared  to 
magnetometer 
systems  in  the 
presence  of  magnetic 
background 

TEM-8  was  unaffected 
by  magnetic 
background. 
Performance  relative 
to  magnetometer 
systems  depends  on 
degree  of  magnetic 
interference. 

1_We  define  the  term  “ordnance  detection”  to  mean  the  percentage  of  ordnance  items  that  produced  electromagnetic  anomalies  discemable  above 
the  noise  floor  and  within  a  defined  search  radius.  The  term  does  not  imply  that  the  anomalies  were  or  were  not  correctly  classified. 

2By  the  term  “false  positive”  we  refer  to  an  EM  anomaly  for  which  no  metallic  conductor  can  be  associated. 

3  SNR  for  both  EM  and  magnetic  systems  is  to  be  calculated  as  the  average  peak  amplitude  of  positive  M3  8  detections  divided  by  the  root  mean 
squared  (RMS)  noise  over  the  entire  target  area. 
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Figure  2.  Map  of  the  FKPBR  site  showing  areas  adjacent  to  Double  Eagle  Airport. 
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Figure  3.  Total  field  magnetic  anomaly  surveyed  by  Sky  Research. 

Airborne  geophysical  systems  do  not  distinguish  between  UXO  and  metallic  scrap.  Map 
products  depict  bombing  targets  (areas  of  high  ordnance  density),  infrastructure  (fences  or  larger 
items  or  areas  of  metallic  debris  associated  with  human  activity),  and  potential  UXO  items 
(discrete  sources).  It  is  not  the  purpose  of  this  test  to  discriminate  between  UXO  and  non-UXO 
items.  Here  the  detection  capability  is  assessed  over  a  large  area  with  110  blind-seeded  items. 
The  predicted  and  resultant  detection  rates  are  listed  in  Table  2. 

Table  2.  Anticipated  detection  rates  for  FKPBR  seeded  grid. 


Ordnance  Type 

Predicted  Detection 

#  of  Seeds 

Achieved  Detection  j 

81  mm 

40% 

12 

100% 

105  mm  projectile 

65% 

8 

100% 

105  mm  HEAT 

60% 

14 

100%  1 

4.2-inch  mortar 

60% 

52 

98% 

155  mm 

80% 

24 

100% 

At  PBR-S12,  the  objective  was  to  assess  the  TEM-8  performance  at  a  site  that  is  not  artificially 
seeded  and  where  in  situ  M3 8  ordnance  (or  at  least  M3 8  fragments)  are  visible  at  the  surface.  It 
is  also  a  site  where  previous  airborne  magnetometer  surveys  have  been  completely  ineffective. 
The  detection  of  M3 8s  at  PBR-S12  was  assessed  by  direct  comparison  with  EM61  survey  results 
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and  validation  digs,  which  are  compared  against  the  anomaly  list  provided  from  the  TEM-8 
survey.  The  expected  Pd  was  90%  for  mostly  intact  M38s  at  PBR-S12.  For  detection  purposes,  a 
“largely  intact-M38”  is  defined  as  any  M38  related  ordnance  or  frag  larger  than  50  x  24  cm,  or 
any  intact  tail-fin.  Actual  performance  is  summarized  in  Table  3. 


Table  3.  Summary  of  targets  detected  by  EM61  and  TEM-8. 


Item 

Excavated 

Detected  by 
EM61 

Detected  by 
TEM-8 

Mostly  intact  M3  8 

38 

37 

36 

Nose  cone 

46 

45 

34 

Initiator 

42 

41 

6 

2"  band 

22 

21 

4 

Wire 

6 

6 

0 

At  PBR-S12,  an  approximation  to  false  alarms  can  be  made  from  the  validation  areas.  Results 
from  a  2002  ORAGS -Arrowhead  magnetometer  survey  provide  a  comparable  magnetic 
database.  Validation  areas  were  selected  from  areas  that  show  concentrations  of  discrete 
anomalies  in  the  TEM-8  data.  Within  these  areas,  EM61  data  were  acquired  and  anomalies  from 
the  EM61  data  were  subsequently  validated  by  excavation.  False  positives  (FP),  defined  as 
anomalies  produced  by  non-metallic  sources,  are  documented.  As  indicated  in  Table  1,  only 
18.7%  of  the  digs  at  PBR-S12  were  airborne  FPs. 

A  quantitative  measure  of  the  improvement  in  SNR  is  a  key  metric  in  terms  of  defining  the 
benefit  of  TEM  over  magnetic  data  obtained  in  areas  where  geologic  fonnations  or  material 
contain  magnetic  minerals  that  influence  background  measurements,  in  this  case  basalt. 
Measurements  were  made  over  PBR-S12  where  both  magnetic  and  electromagnetic  data  have 
been  collected.  SNR  is  described  in  greater  detail  in  Section  7.2.  The  TEM-8  had  significantly 
better  SNR  in  areas  where  moderate  to  severe  basalt-generated  noise  occurred,  four  times  that  of 
the  magnetometer  system  in  the  moderately  basaltic  FKPBR,  and  1300  times  better  than  the 
magnetometer  system  at  the  more  severely  basaltic  PBR-S12.  Where  there  is  less  basalt,  the 
SNR  of  the  TEM-8  was  similar  to  that  of  the  magnetometer  system. 

The  positional  accuracy  is  an  important  constraint  for  enabling  satisfactory  ground  follow-up  of 
anomalies  and  can  be  measured  from  both  the  FKPBR  and  PBR-S12  datasets.  Positional 
accuracy  within  PBR-S12  was  detennined  by  comparing  the  positions  of  detected  and  reported 
anomalies  in  the  PBR-S12  validation  areas  with  the  validation  results.  Positional  accuracy 
within  the  FKPBR  seeded  area  is  determined  by  positions  of  seed  items  provided  by  the  ESTCP 
Program  Office  after  the  “dig  lists”  were  provided  to  ESTCP  by  Battelle.  At  FKPBR,  the  mean 
miss  distance  was  0.34  m  with  a  standard  deviation  of  0.23  m.  At  PBR-S-12  the  mean  miss 
distance  for  successful  digs  associated  with  TEM-8  anomalies  was  0.58  m  with  a  standard 
deviation  of  0.34  m.  The  objective  was  met  at  both  sites. 
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5.0  SITE  DESCRIPTION 


The  two  areas  that  were  selected  for  this  demonstration  are  a  617-acre  portion  of  the  FKPBR 
(Figure  3)  and  a  444-acre  area  surrounding  the  center  of  PBR-S12  (Table  4).  A  100-acre  area 
within  the  FKPBR  area  was  specified  for  emplacement  of  seed  items,  to  be  emplaced  under  the 
direction  of  the  ESTCP  Program  Office  without  involvement  from  AMEC  or  Battelle. 

Table  4.  Breakdown  of  proposed  survey  blocks  for  Battelle  TEM-8  system  at  FKPBR. 


Area 

Altitude 

Purpose 

FKPBR-600 

ALASA* 

To  allow  comparison  of  TEM-8  survey  noise  and  ordnance  detection  to 
airborne  magnetic  data  previously  acquired  by  another  contractor 

FKPBR- 100 

ALASA 

To  assess  TEM-8  sensitivity  to  seeded  items,  emplaced  under  the  direction 
of  the  ESTCP  Program  Office 

PBR-S12 

ALASA 

To  assess  TEM-8  performance  of  detection  of  M38  practice  bombs  in  an 
area  where  basaltic  contamination  is  severe 

* ALASA  -  as  low  as  safely  achievable 


5.1  SITE  LOCATION  AND  HISTORY 

Two  sites  were  selected  for  this  project.  The  ESTCP  Program  Office  had  requested  that  survey 
data  be  acquired  at  the  FKPBR  in  New  Mexico,  where  previous  WAA  surveys  had  been 
conducted  (see  Figure  2). It  was  thought  that  a  demonstration  in  this  area  would  provide  valuable 
comparisons  with  other  WAA  survey  tools  while  reducing  overall  cost  of  the  demonstration  to 
ESTCP. 

Figure  2  depicts  the  two  areas  adjacent  to  Double  Eagle  Airport  that  were  surveyed  in  previous 
ESTCP  WAA  projects.  The  proposed  600-acre  survey  area  is  shown  in  red.  The  blue  box 
within  the  proposed  survey  area  represents  the  100-acre  area  for  emplacement  of  seed  items.  The 
green  circle  is  approximately  where  the  N-3  target  area  is  located.  Locations  of  previous  ground 
surveys  provided  by  M.  May  of  IDA  are  included  as  smaller  rectangles.  The  perimeter  polygons 
for  the  north  and  south  areas  were  provided  by  H.  Nelson  of  ESTCP. 

Figure  3  shows  total  field  magnetic  anomaly  map  of  the  area  at  FKPBR  that  was  surveyed  by 
Sky  Research  in  an  earlier  2005-6  ESTCP  WAA  project.  The  selected  area  in  Figure  3  clearly 
shows  the  geologic  interference  caused  by  basalt  formations  and  float  rocks.  The  inset  shows  the 
location  of  the  100-acrea  area  selected  for  blind-seeding.  The  area  of  interest  shows  generally 
moderate,  and  locally  severe,  geologic  interference  as  a  result  of  basalt  bedrock  and  float 
materials.  The  100-acre  area  was  also  surveyed  by  Sky  Research  in  2009  with  a  newer  airborne 
magnetometer  system  for  ESTCP.  None  of  the  seed  items  were  in  place  when  the  magnetometer 
data  shown  in  Figure  2  were  acquired. 

Survey  data  were  also  acquired  at  the  Kirtland  PBR-S12  target  (S12,  Figure  4).  This  site  was 
chosen  as  being  representative  of  sites  where  ground-based  and  airborne  magnetometer  data  are 
ineffective  for  UXO  mapping  and  detection  due  to  interference  from  basalt  bedrock  and  float 
materials.  Airborne  magnetometer  data  were  acquired  with  the  ORAGS-Arrowhead  system  at 
PBR-S12  and  show  no  distinguishable  response  to  ordnance  at  the  bombing  target,  even  though  a 
concentration  of  M3 8  scrap  can  be  observed  at  the  surface  near  the  center  of  the  target 
(Figure  5). 
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Figure  4.  The  400-acre  TEM-8  survey  area  at  the  Kirtland  PBR-S12  target  (blue  box) 
superimposed  on  the  Arrowhead  magnetic  map  of  the  site,  acquired  in  2002. 

The  area  was  selected  to  allow  comparison  of  magnetic  and  TEM-8  data  and  to  encompass  both 
basaltic  and  non-basaltic  areas  at  PBR-S12.  The  area  selected  is  centered  on  the  target. 
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Figure  5.  Photograph,  looking  north  across  the  center  of  the  PBR  S-12  target. 

The  pile  of  M3  8  scrap  in  the  center  of  the  photograph  was  not  detected  by  the  airborne 

magnetometer  survey. 
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The  FKPBR  site  is  a  38,000-acre  formerly  used  defense  site  (FUDS).  It  has  been  subject  to 
previous  geophysical  surveys  and  partial  excavation,  primarily  under  the  guidance  of  the  ESTCP 
Program  Office.  It  is  currently  undeveloped  open  rangeland.  It  was  used  in  World  War  II  as  a 
training  area  for  Kirtland  AFB.  The  ESTCP  WAA  pilot  study  area  consists  of  5000-6500  acres 
of  land  adjacent  to  Double  Eagle  Airport,  near  Albuquerque,  NM.  There  are  at  least  three 
bombing  targets  and  a  simulated  oil  refinery  target  (SORT)  within  this  study  area.  Known  or 
suspected  ordnance  types  at  the  site  are  M38  practice  bombs  and  250-lb  high  explosive  bombs. 
The  portion  of  the  FKPBR  that  was  surveyed  by  the  TEM-8  includes  the  N-3  target  area  that  is 
just  northwest  of  the  seeded  area  and  two  smaller  targets  adjacent  to  N-3  (see  Figure  3). 

The  Kirtland  PBR-S12  target  is  located  within  land  owned  by  the  Pueblo  of  Laguna  Native 
American  tribe  and  is  a  FUDS  about  35  miles  west  of  Albuquerque,  NM.  The  predominant 
ordnance  type  at  the  site  is  World  War  II  vintage  M38  practice  bombs.  The  Pueblo  of  Laguna 
land  totals  more  than  half  a  million  acres,  and  large  portions  of  this  typically  western  desert 
environment  are  flat  and  devoted  to  ranching.  The  remaining  portions  of  land  are  gently  rolling 
to  nearly  vertical  in  relief  and  were  formed  by  extensive  erosion  of  the  soft,  fine-grained 
underlying  sediments  creating  canyons,  washes,  gullies,  and  arroyos. 

5.2  SITE  GEOLOGY 

The  sites  are  situated  on  the  eastern  edge  of  the  New  Mexico  portion  of  the  Colorado  Plateau, 
east  of  the  Albuquerque-Belen  Basin.  A  series  of  strong  north-south  trending,  high-angle  faults 
separate  the  geologic  provinces,  stepping  downward  from  the  plateau  into  the  basin.  The 
geology  of  the  area  is  dominated  by  both  consolidated  and  unconsolidated  units  and  includes 
sandstone,  mudstone,  claystone,  and  shale.  Igneous  basalt  fonnations  cap  the  mesas  in  the  area 
(e.g.,  Mesa  Lucero,  where  the  PBR-S12  target  is  located).  In  other  locations,  basalts  have 
emanated  from  fissures  or  vents,  providing  sources  for  mafic  alluvium  on  a  more  moderate  scale 
(e.g.,  FKPBR).  The  typical  altitude  is  5000-6000  ft  above  sea  level. 

5.3  MUNITIONS  CONTAMINATION 

With  regard  to  historical  ordnance,  numerous  sites  across  the  entire  area  were  utilized  for  aerial 
bombardment  training  activity.  From  both  visual  inspection  and  previous  Naval  Research 
Laboratory  (NRL)  Multisensor  Towed  Array  Detection  System  (MTADS)  surveys,  the  principal 
ordnance  type  present  at  these  sites  is  the  M3 8  practice  bomb.  Evidence  of  these  ordnance  items 
is  present  on  the  surface  at  all  sites  used  for  this  demonstration,  with  several  hundred  M3 8s 
excavated  during  the  MTADS  demonstration  (McDonald  and  Nelson,  1999). 
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6.0  TEST  DESIGN 


6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  demonstration  has  been  designed  to  address  the  most  common  situation  where  an  airborne 
EM  capability  would  be  beneficial;  that  is  where  background  geology  causes  moderate  to  severe 
perfonnance  issues  for  airborne  magnetometer  systems.  Airborne  systems  are  viewed  primarily 
as  tools  for  WAA  and  as  such  are  not  required  to  achieve  a  high  Pd.  However,  where  geologic 
interference  is  severe,  most  frequently  due  to  basalt,  the  magnetic  response  from  ordnance  may 
be  orders  of  magnitude  smaller  than  the  local  geologic  response.  In  this  project,  we  assess  the 
perfonnance  of  the  TEM-8  system  at  one  site  where  basalt  causes  severe  problems  for 
magnetometer  systems  and  where  M38s  are  the  predominant  ordnance  type,  and  at  a  second  site 
where  geologic  interference  is  less  severe  and  where  a  suite  of  more  challenging  seed  items  were 
emplaced. 

6.2  SITE  PREPARATION 

The  ESTCP  Program  Office  arranged  for  burial  of  110  seed  items  within  the  100-acre  FKPBR 
survey  area.  This  was  conducted  several  months  in  advance  of  the  TEM-8  survey  without 
involvement  from  the  TEM-8  survey  team. 

6.3  SYSTEM  SPECIFICATIONS 

The  Battelle  TEM-8  system  was  described  in  Section  3.1. 

6.4  DATA  COLLECTION 

6.4.1  Scale 

As  discussed  in  Section  5.1,  data  were  acquired  over  a  total  of  1062  acres  at  the  two  sites.  This 
is  a  sufficient  area  for  the  demonstration  of  the  technology  for  WAA  purposes  and  for  allowing 
distribution  of  seed  items  within  an  area  of  sufficient  size  to  be  representative  of  WAA 
applications. 

6.4.2  Sample  Density 

The  TEM-8  system  acquires  data  at  a  line  separation  of  0.75  m  and  a  down-line  sample  interval 
of  a  few  centimeters,  depending  upon  the  system  base  frequency  and  helicopter  flight  speed. 
Once  the  data  are  fully  processed,  a  sample  spacing  of  about  10  cm  is  typical.  The  diameter  of 
the  anomalies  (1.5-2. 5  m  or  larger)  collected  at  acceptable  altitudes  results  in  several 
measurements  per  anomalous  response. 

6.4.3  Quality  Checks 

Methods  utilized  by  Battelle  to  ensure  airborne  survey  success  on  both  current  and  past  airborne 
data  acquisition  surveys  include  daily  quality  assurance  (QA)/quality  control  (QC)  checks  on  all 
system  parameters  (e.g.,  Global  Positioning  System  [GPS],  sensor  operation,  data  recording)  in 
the  acquired  data  sets.  In  addition,  continual  inspection  of  all  system  hardware  and  software, 
ensuring  optimal  performance  during  the  data  acquisition  phase,  and  review  of  data  upon 
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completion  of  each  processing  phase  was  completed.  Data  are  processed,  gridded,  and  reviewed 
daily  to  assure  ongoing  perfonnance.  Lines  were  re-flown  when  the  daily  QC  checks  showed 
gaps  between  lines,  unacceptable  altitude  perturbations,  or  localized  noise. 

6.4.4  Data  Summary 

Data  were  stored  on  the  recording  console  and  transferred  at  least  daily  to  the  desktop  processing 
computer.  Backups  of  data  at  selected  levels  of  processing  were  stored  on  external  hard  drives 
or  DVDs.  The  data  were  then  stored  on  a  large  hard  drive  in  the  Battelle  Oak  Ridge  offices. 
Upon  completion  of  processing,  all  data  including  raw  and  processed  products  were  archived  and 
retained  for  future  reference. 

6.5  VALIDATION 

Validation  of  survey  results  was  conducted  on  both  the  FKPBR  and  PBR-S12  survey  sites.  Seed 
items  at  FKPBR  (Table  2)  were  selected  based  upon  items  that  were  shown  to  be  detectable  at 
Battelle’s  Ohio  UXO  test  grid  under  controlled  conditions.  Without  complete  excavation  of  an 
area,  the  actual  Pd  and  FP  ratio  cannot  be  calculated.  For  FKPBR,  a  Pd  can  be  calculated  for 
seed  items.  The  FP  ratio  cannot  be  calculated  because  the  non-seed  items  were  not  excavated. 
Dig  lists  from  the  airborne  data  were  derived  as  described  later  in  Section  7.  A  search  radius  of 
1.50  m  was  used  to  determine  whether  each  anomaly  constituted  a  positive  hit  corresponding  to 
one  of  the  seed  items.  This  search  radius  and  seed  item  knowledge  provided  the  basis  for 
calculating  the  Pd  and  location  accuracy  of  the  system. 

Five  validation  grids  were  selected  within  the  bounds  of  the  PBR-S12  demonstration  area.  The 
grids  were  located  in  areas  that  contain  a  reasonable  number  of  discrete  anomalies  that  could  be 
surveyed  and  excavated  with  the  allotted  budget.  EM61  ground  surveys  were  conducted  over 
two  of  these  grids,  and  a  dig  list  was  generated  based  upon  those  data.  The  EM61  anomaly  list 
was  edited  to  remove  all  hits  below  a  threshold  equivalent  to  the  TEM-8  picking  threshold.  The 
remaining  anomalies  were  excavated  and  the  Pd,  FP  ratio,  and  location  accuracy  statistics  were 
compiled.  A  search  radius  of  1.50  m  was  used. 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 


7.1  PREPROCESSING 

A  digital  signal  processor  (DSP)  in  the  console  conducts  initial  data  reduction  tasks  prior  to  data 
storage.  Data  responses  are  acquired  at  10.8  kilohertz  (kHz)  sample  rate  and  binned  into  time 
gates.  The  DSP  also  calculates  the  response  values  in  each  of  the  selected  time  gates,  inverts  the 
responses  from  the  negative  transmission  pulses,  and  stacks  between  two  and  six  sets  of  values 
before  storage.  At  270  hertz  (Hz)  base  frequency,  three  time  gates  are  available.  Maps  were 
produced  from  a  single  time  gate  based  on  initial  assessment  of  noise  properties  for  each 
individual  time  gate.  Typically,  Bin  2  is  selected  because  it  maximizes  the  spatial  resolution  of 
the  data  and  the  helicopter  noise  rejection  capabilities.  Additional  processing  steps  are 
conducted  subsequently  to  filter  helicopter  noise,  remove  instrument  drift,  minimize  effects  of 
ground  conductivity,  integrate  base  station  corrected  positioning  data,  and  grid  the  data. 

The  quantity  measured  at  the  receiver  is  the  temporal  rate  of  change  in  the  magnetic  field  as  the 
field  decays  from  its  initial  value  after  the  transmitting  coil  is  turned  off.  Data  are  sampled  at  the 
same  frequency  that  the  transmitter  coil  is  driven  (270  Hz,  225  Hz,  or  90  Hz).  All  data 
processing  is  done  using  Geosoft  Oasis  montaj,  with  the  exception  of  the  GPS  orientation  and 
positioning  data. 

The  position  of  each  receiver  is  calculated  based  on  the  locations  of  two  GPS  antennae,  one  on 
the  starboard  side  and  the  other  on  the  port  side  of  the  helicopter.  Data  streams  from  both  GPS 
antennae  are  post-processed  using  the  base  station  data  to  provide  improved  positioning  and 
orientation  accuracy.  The  orientation  of  the  aircraft  is  calculated  from  the  relative  GPS  locations 
of  the  two  antennae.  The  sensor  locations  are  calculated  based  on  this  aircraft  orientation  and  the 
GPS  location  of  the  antennae  on  the  same  side  of  the  aircraft. 

After  converting  the  raw  data  to  ASCII  and  importing  it  into  Geosoft,  a  low-pass  filter  is  applied 
to  remove  helicopter  rotor  noise.  A  high-pass  filter  is  subsequently  applied  to  remove  effects  of 
aircraft  motion,  vibrations,  and  ground  conductivity. 

7.2  TARGET  SELECTION  FOR  DETECTION 

Anomalies  were  picked  from  the  peaks  of  the  gridded  data  using  a  threshold  based  on  the 
background  noise  levels  for  that  particular  data  set.  The  target  list  was  edited  to  remove  obvious 
artifacts  and  cultural  sources  (fences,  etc).  The  final  list  was  sorted  based  on  target  amplitude. 

Background  noise  is  calculated  as  the  standard  deviation  of  the  gridded  data.  In  data  sets  such  as 
EM  and  magnetic  analytic  signal  that  ideally  have  a  zero-minimum  value,  the  background  noise 
is  the  value  that  responses  must  rise  above  to  be  considered  anomalous.  The  background  noise  is 
an  aggregate  of  all  noise  measured  by  the  TEM-8  sensors  at  the  site,  incorporating  random  and 
systematic  sources  such  as  the  electronics,  platform,  local  geology,  and  altitude  attenuation.  This 
background  value  is  unbiased  by  the  number  and  amplitude  of  anomalies  unless  the  area  is  so 
contaminated  that  there  is  little  or  no  background  to  be  measured.  The  amplitude  threshold  used 
in  anomaly  selection  is  some  multiple  of  this  value  and  represents  the  minimum  SNR  of  the 
anomaly  list.  The  optimal  threshold  may  vary  depending  on  the  size  of  the  sample  set  (grid)  and 
the  distribution  about  the  mode  (geologic  noise  level).  In  benign  geologies,  an  SNR  of  2  may  be 
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acceptable.  In  other  environments  an  SNR  of  10  may  be  preferred.  The  area  chosen  to  calculate 
the  noise  should  be  as  large  as  possible,  as  long  as  background  conditions  (geology,  altitude, 
equipment)  remain  consistent.  The  threshold  for  picking  anomalies  at  PBR-S12  was  deliberately 
set  low  so  that  perfonnance  of  low  amplitude  anomalies  could  be  tested. 

7.3  PARAMETER  ESTIMATES 

This  demonstration  is  centered  on  detection  capabilities  of  the  airborne  system.  The  target 
location  is  the  only  parameter  being  estimated  from  this  data  set.  The  target  location  is 
determined  directly  from  the  peak  amplitude  of  the  anomaly  in  the  gridded  data  set.  As  such,  the 
target  location  resolution  is  limited  to  the  grid  cell  size.  No  inversion  has  been  perfonned  to 
estimate  other  target  parameters. 

7.4  CLASSIFIER  AND  TRAINING 

The  calibration  line  results  were  used  to  establish  thresholds  for  anomaly  selection  and  data 
filtering.  EM  anomalies  were  classified  strictly  by  response  amplitude,  with  anomalies  thought 
to  be  caused  by  cultural  features  excluded.  To  study  the  relationship  of  EM  and  magnetic 
anomalies,  areas  where  both  types  of  data  occur  were  evaluated.  A  detennination  was  performed 
selecting  the  closest  magnetic  anomaly  to  a  particular  EM  anomaly.  If  the  EM  and  magnetic 
anomalies  were  spatially  close  enough  to  one  another  it  was  assumed  that  they  were  likely  to  be 
produced  by  the  same  body.  A  scatter  plot  was  then  produced  showing  the  amplitude  of  the 
magnetic  anomalies  plotted  against  the  EM  anomalies.  There  is  currently  no  reliable  method  for 
discriminating  fragments  from  intact  ordnance  using  TEM-8  data. 

7.5  DATA  PRODUCTS 

This  section  provides  the  summary  data  products  for  the  report.  High  resolution  maps  have  been 
provided  to  the  ESTCP  Program  Office  along  with  anomaly  lists  and  Geosoft  databases. 

7.5.1  Test  Grid  Results 

A  test  grid  consisting  of  two  parallel  lines  was  established  near  the  Double  Eagle  General 
Aviation  Airport  located  near  Albuquerque,  NM,  in  an  area  that  was  within  a  few  meters  of  the 
2007  demonstration  test  grid  of  the  Battelle  vertical  (magnetic)  gradient  (VG)-16  and  VG-22 
systems  (Battelle,  2008a).  Ordnance  that  were  emplaced  on  the  grid  were  provided  by  Battelle, 
Kirtland  AFB,  and  the  U.S.  Army  Engineering  Support  Center,  Huntsville  (USAESCH). 
USAESCH  provided  two  4.2-inch  mortars,  a  105-HEAT  round  with  tail,  and  a  105  mm 
projectile.  Battelle  provided  two  81  mm  mortars,  a  105-HEAT  without  tail,  a  105  mm  projectile, 
two  155  mm  projectiles,  two  60  mm  mortars  with  tails,  and  M3 8s  (one  largely  intact,  one  intact 
but  without  fins,  a  tail  section,  and  a  pair  of  primer  caps).  Four  test  items,  unrelated  to  the 
ESTCP  demonstration  were  provided  by  Kirtland  AFB  and  emplaced  in  the  grid.  They  included 
an  82  mm  Russian  projectile  with  tail,  a  120G  round,  an  MK35,  and  a  155  FWD  mortar.  The 
test  grid  was  flown  daily,  typically  at  the  beginning  of  the  survey  day,  and  additionally  was 
flown  at  a  suite  of  altitudes.  A  representative  map  of  the  Test  Grid  results  is  shown  on  Figure  6. 
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Figure  6.  Representative  example  of  FKPBR  TEM-8  test  grid. 

Amplitudes  of  the  anomalies  associated  with  each  of  the  test  items  for  every  test  flight  were 
compiled  and  plotted.  These  were  used  to  assess  representative  amplitudes  for  each  type  of  test 
item  as  a  function  of  altitude.  In  turn,  these  amplitudes  were  used  to  select  anomaly  thresholds 
for  data  from  the  FKPBR  and  PBR-S12  sites.  A  summary  plot  of  these  data  is  provided  in 
Figure  7. 
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Figure  7.  TEM-8  amplitude  as  a  function  of  altitude  for  items  in  the  test  grid. 

Each  point  represents  a  measurement  from  a  separate  pass  over  each  of  the  items,  including  the 
daily  flyovers  and  a  set  of  flights  at  selected  altitudes  that  were  conducted  early  in  the  field 
deployment. 

7.5.2  FKPBR  Results 

The  TEM-8  results  of  the  FKPBR  survey  area  are  depicted  in  Figure  8.  The  area  surveyed 
encompassed  a  total  of  617  acres.  The  N-3  target  area  (located  at  330750  mE,  3893900  mN, 
Universal  Transverse  Mercator  [UTM]  Zone  13N)  and  several  of  the  nearby  secondary  targets 
are  apparent  to  the  northwest  of  the  blind-seeded  area  in  Figure  8.  The  density  of  anomalies  falls 
off  to  the  east  and  southeast  of  the  N-3  target.  Power  line  interference  can  be  seen  along  the 
northern  boundary.  The  mean  laser  altimeter  altitude  from  the  entire  617-acre  area  was  1.23  m. 
Average  altitude  from  the  laser  altimeter  within  the  100-acre  seeded  test  grid  at  FKPBR  was  0.97 
m.  The  mean  sensor  altitude  within  the  same  area  was  1.3  m. 
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Figure  8.  TEM-8  survey  results  from  the  FKPBR  area. 

N-3  target  area  and  two  smaller  target  areas  are  located 
northwest  of  the  seeded  area  (red  box). 

An  expanded  view  of  the  blind-seeded  area  at  FKPBR  is  depicted  in  Figure  9.  A  general 
southeastward  decay  in  anomaly  density  is  observed  in  Figure  10  and  is  presumably  associated 
with  the  N3  target,  along  with  some  concentration  of  anomalies  associated  with  the  blind-seeded 
items. 


Based  on  the  anomaly  amplitudes  for  the  four  types  of  items  emplaced  in  the  blind-seeded  grid, 
the  anomalies  were  divided  into  three  categories:  A,  B,  and  C.  Types  of  ordnance  that  might 
cause  anomalies  in  each  amplitude  category  were  projected  based  on  these  amplitude  results.  A 
total  of  1292  picks  were  submitted  with  477  in  the  “A”  category,  344  in  the  “B”  category,  and 
471  in  the  “C”  category.  Depth  of  burial  is  assumed  to  be  less  than  0.3  m  and,  if  deeper,  could 
affect  the  breakdown  of  anomalies  in  the  three  categories  described. 
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Figure  9.  TEM-8  map  of  the  blind-seeded  area  at  FKPBR. 

Targets  labeled  as  high  priority  on  the  map  and  “A”  on  the  target  list  are  represented  by  a  large 
circle  and  are  4.3  parts  per  million  (ppm)  and  higher.  Targets  labeled  as  medium  priority  on  the 
map  and  “B”  on  the  target  list  are  represented  by  a  medium-sized  circle  and  are  between  1 .3  ppm 
and  4.3  ppm.  Targets  labeled  as  low  priority  on  the  map  and  “C”  on  the  target  list  are 
represented  by  a  small  circle  and  are  below  1.3  ppm  and  above  0.9  ppm.  However,  visual 
inspection  was  done  resulting  in  ranking  of  some  higher  and  some  lower  on  the  basis  of  the 
profile  and  map  character. 

7.5.3  PBR  S12  Results 

The  total  area  flown  at  PBR  S12  was  444  acres.  Before  the  survey  began,  it  was  noted  that  the 
study  area  extended  beyond  the  edge  of  Mesa  Lucero.  This  resulted  in  a  small  portion  of  the 
proposed  area  that  could  not  be  surveyed.  To  compensate  for  this,  additional  lines  were  flown 
along  the  northern  boundary  of  the  proposed  area,  ultimately  leading  to  an  11%  increase  in  the 
total  survey  area.  The  plan  view  of  the  survey  area  is  shown  in  Figure  10.  The  edge  of  the  mesa 
is  apparent  in  the  southeast  quadrant  of  the  map  where  data  were  not  taken. 
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Figure  10.  TEM-8  survey  area  at  PBR  S12. 

Five  areas  were  selected  for  possible  ground-based  mapping  and  validation,  based  on  the 
airborne  survey  results.  These  five  areas  (numbered  1  through  5  on  Figure  10)  were  chosen  due 
to  the  modest  number  of  large  anomalies  and  because  the  overall  density  of  anomalies  was  lower 
than  in  the  central  target  area,  indicating  that  there  were  few  overlapping  anomalies  in  the  data. 
From  these  five  areas,  AMEC  selected  two,  Areas  1  and  4,  for  validation.  Figure  1 1  depicts  the 
TEM-8  maps  for  Area  1  at  PBR-S12. 
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Figure  11.  TEM-8  map  for  Area  1  at  PBR-S12. 

Symbols  on  the  maps  are  associated  with  anomalies  picked 
for  three  different  ranges  of  amplitudes. 

Dig  lists  were  prepared  and  are  listed  by  dig  priority.  The  dig  priority  is  represented  by  different 
symbols  in  Figure  1 1 .  Thresholds  for  Grids  1  and  4  were  selected  on  the  basis  of  measurements 
of  M38s  and  M38  scrap  at  the  Kirtland  Test  Grid  (Section  7.5.1).  The  priority  “A”  anomalies  are 
considered  high  priority  and  are  represented  on  the  map  with  an  “X”.  Circles  are  used  to 
represent  medium  priority  targets,  which  are  also  likely  UXO  fragments  or  munitions  and 
explosives  of  concern  (MEC),  and  are  appropriate  for  investigation.  The  “C”  anomalies, 
represented  by  a  “+”  are  thought  to  be  predominantly  noise.  The  thresholds  between  the  A,  B, 
and  C  categories  are  generally  2.15  and  1.08  ppm  respectively,  but  after  reviewing  each 
anomaly,  some  have  been  ranked  higher  and  some  lower  on  the  basis  of  their  profile  and  map 
character.  A  lower  bound  of  0.65  ppm  was  placed  on  the  “C”  anomalies.  The  total  number  of 
anomalies  selected  in  each  of  the  three  categories  is  listed  in  Table  5. 
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Table  5.  Numbers  of  anomalies  picked  for  each  of  the  five  validation  areas  at  PBR  S12. 


Area 

Priority  A 

Priority  B 

Priority  C 

Total 

1 

41 

33 

62 

136 

2 

24 

22 

60 

106 

i  3 

13 

18 

70 

101 

4 

34 

40 

52 

126 

5 

36 

44 

55 

135 

An  altitude  map  for  the  study  area  at  S-12  is  shown  in  Figure  12.  The  mean  altitude  at  the  S- 12 
survey  site  as  a  whole  was  3.73  m.  In  the  north,  altitudes  tended  to  be  higher  due  to  vegetation 
(Figure  12).  Mean  altitudes  from  the  laser  altimeter  for  Area  1  was  2.016  m,  and  for  Area  4  was 
2.094  m.  Mean  sensor  altitudes  are  generally  about  0.3  m  higher  than  the  laser  altitudes. 


Figure  12.  Altitude  map  of  the  entire  S-12  study  area. 

Note  higher  altitudes  in  tree-covered  areas  in  the  north  and  a  linear  north  trending  feature 
associated  with  a  fence  in  the  western  half  of  the  study  area. 
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8.0  PERFORMANCE  ASSESSMENT 


Effectiveness  of  the  demonstration  is  detennined  from  comparisons  of  the  processed  and 
analyzed  results  from  the  demonstration  survey  and  the  established  ground  truth.  Some 
qualitative  parameters  may  be  judged  against  results  of  previous  airborne  and  ground-based 
surveys  at  FKPBR  and  elsewhere.  Evaluation  of  seeded  items  provides  a  basis  for  assessing 
detection  of  small  ordnance  items.  These  comparisons  include  both  the  quantitative  and 
qualitative  items  described  in  this  section  that  are  documented  fully  in  project  reports  available 
from  ESTCP.  Demonstration  success  is  defined  as  the  successful  acquisition  of  airborne 
geophysical  data  (without  any  aviation  incident  or  airborne  system  failure)  and  meeting  the 
baseline  requirements  for  system  perfonnance,  as  established  previously  in  Section  4.1. 
Methods  utilized  by  Battelle  on  both  current  and  past  airborne  acquisitions  to  ensure  airborne 
survey  success  include  daily  QA/QC  checks  on  all  system  parameters  (e.g.,  GPS,  magnetometer 
operation,  data  recording,  system  compensation  measurements)  in  the  acquired  data  sets,  a  series 
of  compensation  flights  at  the  beginning  of  each  survey,  continual  inspection  of  all  system 
hardware  and  software  ensuring  optimal  performance  during  the  data  acquisition  phase,  and 
review  of  data  upon  completion  of  each  processing  phase. 

Data  collection  occurred  at  the  specified  flight  altitudes  over  the  various  test  areas.  Table  1 
identified  the  expected  perfonnance  criteria  for  this  project,  complete  with  expected/desired 
values  (quantitative)  and/or  definitions  and  descriptions  (qualitative). 

8.1  FKPBR  SEEDED  AREA 

After  the  AMEC/Battelle  team  submitted  the  target  list  to  ESTCP  for  the  FKPBR  seeded  area, 
the  IDA  conducted  a  comparison  of  the  lists  with  the  actual  locations  of  seed  items.  In  all,  110 
items  were  emplaced  in  the  100-acre  grid.  Validation  results  compiled  by  IDA  are  listed  in 
Table  6. 

8.1.1  Detection 

The  performance  results  demonstrate  that  the  TEM-8  system  was  successful  in  detecting  109  out 
of  the  110  seed  items  for  a  1.5  m  search  radius  and  108  out  of  1 10  for  a  1  m  search  radius.  The 
one  missed  item  was  a  4.2-inch  mortar,  which  was  1.51  m  from  the  nearest  item  on  the  dig  list. 
The  amplitude  response  of  the  seed  items  was  larger  than  anticipated  from  the  test  grid.  All  but 
four  of  the  seeded  items  had  amplitudes  in  the  “A”  category,  exceeding  4.3  ppm.  The  other  four 
items  were  all  in  the  “B”  category,  and  all  had  amplitudes  exceeding  1.3  ppm.  The  amplitude 
variance  is  thought  to  be  related  to  the  nature  of  flights  over  the  test  grid.  Although  test  items 
were  detected  in  flights  over  the  test  grid,  there  was  often  a  lateral  offset  (greater  than  the  0.375 
m  maximum  offset  if  ordnance  is  within  the  receiver  array)  between  sensors  and  targets, 
resulting  in  lower  measured  amplitudes.  Such  offsets  do  not  generally  occur  when  the  system  is 
conducting  a  full  density  survey. 
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Table  6.  Dig  results  for  the  blind-seeded  area  at  FKPBR.  (Mean  and  standard  deviation  of  miss  distances  in  meters) 


Halo 

uxo 

Total  # 

#  Seeds 

Mean 

Mean 

Std  Dev 

Std  Dev 

Mean 

Std  Dev 

Radius 

Type 

Seeds 

Detected 

Pd 

(Xi) 

(Yi) 

(Xi) 

(Yi) 

(miss  dist) 

(miss  dist) 

0.5  m 

All  UXO 

110 

87 

0.79 

0.02 

0.00 

0.22 

0.16 

0.24 

0.13 

0.5  m 

105  mm  P 

8 

5 

0.63 

-0.14 

0.03 

0.18 

0.06 

0.18 

0.14 

0.5  m 

4.2  inches 

52 

38 

0.73 

0.03 

0.02 

0.21 

0.19 

0.25 

0.12 

0.5  m 

155  mm  P 

24 

22 

0.92 

0.07 

-0.02 

0.22 

0.11 

0.22 

0.11 

0.5  m 

81  mm  M 

12 

9 

0.75 

0.06 

0.03 

0.29 

0.19 

0.29 

0.19 

0.5  m 

105H 

14 

13 

0.93 

-0.06 

-0.04 

0.19 

0.19 

0.25 

0.10 

1  m 

All  UXO 

110 

108 

0.98 

0.09 

-0.02 

0.35 

0.17 

0.33 

0.22 

1  m 

105  mm  P 

8 

8 

1.00 

0.22 

-0.04 

0.52 

0.11 

0.42 

0.37 

1  m 

4.2  inches 

52 

50 

0.96 

0.14 

-0.01 

0.34 

0.20 

0.35 

0.21 

1  m 

155  mm  P 

24 

24 

1.00 

0.02 

-0.01 

0.28 

0.11 

0.25 

0.15 

1  m 

81  mm  M 

12 

12 

1.00 

-0.02 

0.01 

0.43 

0.18 

0.38 

0.24 

1  m 

105H 

14 

14 

1.00 

0.01 

-0.04 

0.33 

0.18 

0.30 

0.22 

1.5  m 

All  UXO 

110 

109 

0.99 

0.09 

-0.02 

0.36 

0.17 

0.34 

0.23 

1.5  m 

105  mm  P 

8 

8 

1.00 

0.22 

-0.04 

0.52 

0.11 

0.42 

0.37 

1.5  m 

4.2  inches 

52 

51 

0.98 

0.16 

-0.02 

0.36 

0.20 

0.37 

0.23 

1.5  m 

155  mm  P 

24 

24 

1.00 

0.02 

-0.01 

0.28 

0.11 

0.25 

0.15 

1.5  m 

81  mm  M 

12 

12 

1.00 

-0.02 

0.01 

0.43 

0.18 

0.38 

0.24 

1.5  m 

105H 

14 

14 

1.00 

0.01 

-0.04 

0.33 

0.18 

0.30 

0.22 

M  =  mortar,  P=projectile,  h=howitzer,  Std  Dev  =  standard  deviation,  miss  dist  =  miss  distances  in  meters,  Xi  =  easting  offset  in  meters,  Yi  =  northing  offset  in  meters,  m  =  meters,  mm=millimeters, 


Pd  =  probability  of  detection 


All  109  detected  test  items  were  among  the  first  565  listed  on  the  prioritized  dig  list.  The 
remaining  727  anomalies  on  the  dig  list,  categorized  as  false  positives  on  the  pseudo-receiver 
operating  characteristic  (ROC)  curve,  are  of  unknown  origin.  Because  the  100-acre  blind-seeded 
area  lies  within  300  m  of  the  center  of  the  N3  target  and  overlaps  some  of  the  previously 
identified  ancillary  targets,  it  is  probable  that  a  large  portion  of  these  727  anomalies  are  also 
ordnance  related.  This  cannot  be  determined  without  intrusive  validation.  A  pseudo-ROC  curve 
for  the  FKPBR  is  provided  in  Figure  13. 


0  200  400  600  800  1000  1200 

"False"  Anomaly  Count 


Figure  13.  Pseudo-ROC  curve  for  the  FKPBR  blind-seeded  area. 

The  false  anomalies  are  of  unknown  origin,  and  given  site  conditions, 
it  is  very  likely  that  many  of  them  are  ordnance-related. 

8.1.2  Positional  Accuracy 

Positional  accuracy  for  the  FKPBR  site  was  much  better  than  required,  based  on  the  guidelines 
established  in  advance  of  the  survey  (Figure  14).  The  mean  miss  distance  was  0.34  m  with  a 
standard  deviation  of  0.23  m.  The  missed  4.2-inch  mortar  is  included  in  this  figure  for 
completeness,  near  the  eastern  axis  of  the  plot  at  1.51  m  offset,  1  cm  from  inclusion  as  a  “hit.” 
The  positioning  errors  are  clearly  skewed  in  an  east-west  direction,  perpendicular  to  the  flight 
path.  This  may  be  attributed  to  the  larger  sample  interval  in  that  direction,  determined  by  the 
receiver  coil  spacing,  whereas  the  on-line  measurements  occur  at  intervals  of  approximately 
0.1m. 
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Figure  14.  Positional  accuracy  for  the  FKPBR  blind-seeded  targets. 

8.1.3  Noise  Assessment  at  FKPBR 

A  comparison  of  TEM-8  and  airborne  magnetometer  SNR  was  performed,  using  grids  from  the 
TEM-8  data  and  2005  Sky  Research  “Helimag”  data  provided  by  ESTCP.  For  both  the  TEM-8 
and  Helimag  data  sets,  two  areas  were  chosen  for  assessment.  The  western  area  was  selected  as 
representative  of  a  quiet  magnetic  environment,  while  the  eastern  block  was  representative  of  a 
noisier  magnetic  environment.  There  are  no  blind-seeded  anomalies  within  either  block.  The 
average  signal  in  the  entire  600-acre  area  was  estimated  by  taking  the  average  of  the  peak  of  770 
coincident  anomalies  (within  1  m)  picked  from  the  Helimag  and  TEM-8  gridded  data.  The 
standard  deviation  of  the  profile  data  was  calculated  within  each  of  the  blocks  for  both  data  sets 
to  provide  a  measure  of  noise.  Noise  measures  were  similar  for  the  TEM-8  data  in  the  two 
blocks  but  varied  by  a  factor  of  about  5  for  the  magnetometer  data  (Table  7).  Overall,  the  TEM- 
8  exhibited  an  SNR  of  0.81  times  that  of  Helimag  in  the  quiet  area  and  4.0  times  that  of  Helimag 
in  the  noisy  area. 


Table  7.  SNR  estimates  for  the  FKPBR  study  area. 


TEM-8-West 

TEM-8  East 

Helimag  West 

Helimag  East 

Estimated  noise 

0.26  ppm 

0.29  ppm 

1.27  nT/m 

6.78  nT/m 

Estimated  signal 

11.27  ppm 

11.27  ppm 

66.81  nT/m 

66.81  nT/m 

SNR 

42.67 

39.31 

52.48 

9.86 

ppm  =  parts  per  million,  nT/m  =  nanoteslas  per  meter 


Magnetometer  data  were  also  acquired  in  2009  with  the  seed  items  present  at  the  FKPBR  site  for 
ESTCP  by  Sky  Research  (Wright  and  Fonda,  2009).  The  results  of  that  survey  are  not  presented 
here,  but  should  be  available  in  other  ESTCP  reports  and  in  UXO  Forum  online  presentations. 
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8.2  PBR  S12  VALIDATION 


8.2.1  EM61  Data  and  Selection  of  Anomalies  for  Validation 

EM61  data  were  acquired  in  Grids  1  and  4  at  PBR-S12  for  validation  of  the  TEM-8  data.  Results 
for  Grid  1  are  shown  in  Figure  10.  EM61  anomalies  were  picked  in  three  groups,  selected  to 
correlate  with  the  three  TEM-8  categories.  The  three  groups  were  based  on  measured  TEM-8 
and  EM61  results  from  the  calibration  grid.  The  EM61  results  represent  a  standard  EM61-MK2, 
bottom  coil,  gate  1.  The  resulting  threshold  conversions,  based  on  these  measurements,  are 
provided  in  Table  8. 

Table  8.  Threshold  values  for  TEM-8  and  equivalent  thresholds  for  EM61. 


Anomaly 

Group 

TEM-8 

Amplitude 

Range 

EM61 

Equivalent 

Range 

A 

>  2.16  ppm 

>100  mV 

B 

1.08-2.16  ppm 

40-100  mV 

C 

<  1.08  ppm 

<40  mV 

ppm  =  parts  per  million,  mV  =  millivolts 


8.2.2  Detection  Results 

From  the  EM61  surveys,  222  anomalies  were  picked  in  Grid  1  (Figure  15)  and  180  anomalies  in 
Grid  4  (Table  9).  From  the  two  lists,  all  of  the  TEM-8  Category  “A”,  most  of  the  “B”  anomalies 
and  half  or  more  of  the  “C”  anomalies  were  selected  for  excavation.  Total  number  of  digs  in 
Grid  1  was  168  (EM61  Digs  +  TEM-8  Digs  -  Overlap),  and  in  Grid  4  was  157.  These  are 
tabulated  in  Table  9.  All  EM61  anomalies  in  both  grids  yielded  detections;  14  of  the  TEM-8 
digs  in  Grid  1  and  22  in  Grid  4  yielded  false  positives.  Figure  16  shows  the  breakdown  of  TEM- 
8  performance  for  all  excavated  anomalies  (whether  selected  from  TEM-8  or  EM61  dig  lists)  by 
weight.  All  but  two  of  the  excavated  items  over  5  lb  were  detected  by  TEM-8,  and  78%  of  the 
excavated  items  weighing  between  1  and  5  lb  were  detected.  TEM-8  detected  31%  of  the  M38 
frag  that  weighed  less  than  1  lb. 

Table  9.  Dig  results  for  PBR  S12  grids. 

TEM-8  picks  are  further  divided  into  the  A,  B,  and  C  categories. 

Numbers  of  anomalies  picked  by  both  EM61  and  TEM-8  are  listed  as  “overlap.” 


Source  Picks  Digs  Detects 

EM61 

222 

143 

143 

TEM-8 

136 

100 

86  ! 

TEM-8  A 

41 

41 

39 

TEM-8  B 

32 

32 

28  ! 

;  TEM-8  C 

63 

27 

19 

Overlap 

77 

75 

75  ! 

Source  Picks  Digs  Detects 

EM61 

180 

129 

129  ' 

TEM-8 

126 

93 

71 

TEM-8  A 

34 

34 

34 

TEM-8  B 

39 

39 

31 

TEM-8  C 

52 

20 

6 

Overlap 

69 

66 

66  | 

33 


S12  Grid  1 

Selected  Anomalies 


10 


(meters) 

WGS  84  /  UTM  zone  13N 


20 


Air+  EM61  Targets  X 
Air  Target  O 

EM61  Targets  + 
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Figure  15.  EM61  map  of  Grid  1  at  PBR-S12. 
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Figure  16.  Graphical  representation  of  the  distribution  of  detects  to  total  picks  as  a 

function  of  weight. 

Four  weight  ranges  are  shown,  and  these  are  clearly  not  linear  in  the  range  of  weights.  Detection 
falls  off  sharply  for  items  smaller  than  one  pound,  but  it  is  noteworthy  that  some  of  the  low  mass 
items  are  even  detected. 

Table  10.  Summary  of  targets  detected  by  EM61  and  TEM-8. 


Excavated 

Detected  by 
EM61 

Detected  by 
TEM-8  ' 

Mostly  intact  M3  8 

38 

37 

36 

Nose  cone 

46 

45 

34  j 

Initiator 

42 

41 

6 

2-inch  band 

22 

21 

4 

Wire 

6 

6 

0 

8.2.3  Positional  Accuracy 

The  positional  accuracy  of  the  TEM-8  system  at  PBR-S-12  is  shown  in  Figure  17.  The  mean 
miss  distance  for  successful  digs  associated  with  TEM-8  anomalies  was  0.58  m  with  a  standard 
deviation  of  0.34  m.  There  were  two  digs  outside  the  1.5  m  search  radius.  The  distribution  of 
positioning  errors  for  PBR-S12  is  larger  N-S  than  E-W  and  skewed  opposite  to  the  direction  that 
it  was  skewed  for  FKPBR  (Figure  17)  because  the  flight  direction  was  shifted  about  90  degrees. 
This  larger  standard  deviation  is  not  fully  understood  but  could  be  related  to  the  way  that  the 
locations  of  excavated  targets  were  measured,  the  accuracy  of  positioning  for  those  items,  or 
other  factors. 
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Figure  17.  Positional  errors  for  PBR-S12  validations. 

8.2.4  Noise  Assessment  at  PBR-S12 

A  comparison  of  TEM-8  SNR  with  ORAGS-Arrowhead  airborne  magnetometer  SNR  was 
performed,  using  data  acquired  in  2002  by  the  Battelle  team  while  they  were  employed  at  Oak 
Ridge  National  Laboratory.  Because  no  anomalies  could  be  recognized  in  the  total  field 
magnetometer  data  from  the  PBR-S12  target,  the  value  for  signal  was  established  from 
measurements  of  a  largely  intact  M38.  We  used  the  same  M38  as  a  test  item  for  the  TEM-8  test 
grid  at  FKPBR  and  for  the  VG  system  at  Twentynine  Palms,  CA,  in  January  2008.  The  TEM-8 
response  to  this  M38  was  67.3  ppm,  and  the  analytic  response  (taken  from  the  gridded  analytic 
signal  data  from  the  Twentynine  Palms  test  grid)  was  46.6  nanoteslas  per  meter  (nT/m). 

At  PBR-S12,  a  2.6-acre  area  was  selected  for  noise  assessment  based  on  the  sparseness  of  TEM- 
8  anomalies  in  that  area  and  distance  from  the  target  center.  As  before,  the  standard  deviation  of 
the  profile  data  was  calculated  within  each  of  the  blocks  for  both  data  sets  to  provide  a  measure 
of  noise.  Based  on  these  estimations,  the  SNR  for  TEM-8  and  ORAGS-Arrowhead  systems  at  S- 
12  were  determined,  as  summarized  in  Table  11. 

Table  11.  SNR  comparison  for  an  M38  at  PBR-S12. 


TEM-8 

ORAGS-Arrowhead 

Estimated  noise 

0.23  ppm 

204.5  nT/m 

Estimated  signal 

67.3  ppm 

46.6  nT/m 

SNR 

292.7 

0.23 

nT/m  =  nanoteslas  per  meter  ,  ppm  =  parts  per  million 


The  TEM-8  SNRs  for  M38s  at  PBR-S12  are  larger  than  those  reported  at  FKPBR  because  of  the 
large  response  generated  by  M38s  in  both  magnetic  and  electromagnetic  systems.  The  extreme 
improvement  in  SNR  for  TEM-8  over  the  magnetometer  system  is  due  primarily  to  the  very 
large  noise  levels  that  occur  with  magnetometers  over  basalts.  The  important  result  here  is  the 
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ratio  of  TEM-8  SNR  to  Arrowhead  SNR,  nearly  1300  times  better,  which  is  representative  of  the 
improvement  that  can  be  expected  when  using  electromagnetic  systems  rather  than 
magnetometer  systems  in  severe  basaltic  environments. 

8.3  DISCUSSION 

By  all  measures,  the  performance  of  the  TEM-8  at  FKPBR  and  PBR-S12  exceeded  expectations, 
as  indicated  by  the  performance  metrics.  Nearly  all  blind-seeded  items  were  detected  with 
amplitudes  that  were  well  above  the  noise  floor,  suggesting  that,  in  hindsight,  it  would  have  been 
better  to  have  emplaced  smaller  items,  such  as  60  mm  mortars  in  addition  to  the  ones  that  were 
seeded.  Test  grid  results  (Figure  8),  which  included  TEM-8  measurements  of  60  mm,  support  a 
view  that  some  portion  of  these  would  have  been  detected  had  they  been  included.  The  PBR- 
S12  results  demonstrated  the  ability  of  the  system  to  detect  a  portion  of  the  frag  smaller  than 
1  lb. 

In  both  areas,  the  system  also  appears  to  indicate  lower  numbers  of  false  positives  than  typically 
seen  with  magnetometer  systems.  At  the  FKPBR  blind-seeded  grid,  109  seeded  items  were 
detected  within  the  first  565  prioritized  digs.  In  this  case,  many,  if  not  most  of  the  unknown 
anomalies  are  likely  associated  with  ordnance  or  frag  associated  with  the  periphery  of  the  N-3 
target  or  its  satellite  targets.  Figure  9,  which  shows  the  TEM-8  map  of  the  617-acre  area  at 
FKPBR,  shows  concentrations  of  anomalies  that  appear  to  be  like  those  of  bombing  targets, 
which  seem  to  continue  into  the  100-acre  blind-seeded  area.  Similarly,  approximately  81%  of 
the  TEM-8  anomalies  that  were  excavated  in  Grids  1  and  4  at  PBR-S12  were  shown  to  be 
associated  with  ordnance  or  frag. 

In  detennining  the  suitability  of  an  area  for  TEM-8  operation,  one  of  the  most  critical  factors  is 
the  altitude  above  ground  surface  that  the  site  may  be  flown.  Figure  12  provides  a  basis  for 
estimating  the  sensitivity  of  the  system  to  various  types  of  ordnance  as  a  function  of  altitude. 
The  results  of  this  study  indicate  that  TEM-8  should  be  a  valuable  tool  for  WAA  surveys, 
particularly  in  those  areas  where  geologic  interference  is  problematic  for  magnetometer  systems. 
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9.0  COST  ASSESSMENT 


The  cost  of  an  airborne  survey  depends  on  many  factors,  including: 

•  Helicopter  service  costs,  which  depend  on  the  cost  of  ferrying  the  aircraft  to  the 
site,  fuel  costs,  terrain,  and  vegetation  conditions  impacting  flight  line 
configuration,  turnaround,  etc. 

•  Total  size  of  the  blocks  to  be  surveyed 

•  Length  of  flight  lines 

•  Extent  of  topographic  irregularities  or  vegetation  that  can  influence  flight 
variations  and  performance 

•  Ordnance  objectives,  which  dictate  survey  altitude  and  number  of  flight  lines 

•  Temperature  and  season,  which  control  the  number  of  hours  that  can  be  flown 
each  day 

•  Location  of  the  site,  which  can  influence  the  cost  of  logistics 

•  Survey  objectives  and  density  of  coverage,  specifically  high  density  for  individual 
ordnance  detection  versus  transects  for  target/impact  area  delineation  and 
footprint  reduction 

•  Swath  width  and  continuity — some  systems  require  interleaving  for  full  coverage, 
and  hence  can  require  more  flying  than  others. 

The  difference  in  cost  for  the  TEM-8  electromagnetic  and  VG-16  vertical  magnetic  gradient 
systems  lies  largely  in  their  swath.  The  VG-16  system  acquires  data  along  an  entire  12  m  swath 
with  each  pass,  while  the  TEM-8  requires  twice  as  many  flight  passes  to  cover  the  same  area. 
This  causes  the  acquisition  cost  to  be  nearly  double  for  the  TEM-8. 

9.1  COST  MODEL 

Cost  information  associated  with  the  demonstration  of  the  TEM-8  airborne  technology  was 
closely  tracked  and  documented  before,  during,  and  after  the  demonstration  to  provide  a  basis  for 
detennination  of  the  operational  costs  associated  with  this  technology.  It  is  important  to  note 
that  the  costs  for  airborne  demonstrations  and  surveys  are  very  much  dependent  on  the  character, 
size,  and  conditions  at  each  site;  ordnance  objectives  of  the  survey  (e.g.,  flight  altitude);  type  of 
survey  conducted  (e.g.,  high-density  or  transects);  and  technology  employed  for  the  survey 
(e.g.,  total  field  magnetic,  vertical  magnetic  gradient,  time  domain  electromagnetic  induction)  so 
that  a  universal  formula  cannot  be  fully  developed.  These  costs  include  both  operational  and 
equipment  costs  associated  with  system  application;  mobilization  and  demobilization  of 
equipment  and  personnel;  salary  and  travel  costs  for  project  staff;  subcontract  costs  associated 
with  helicopter  services,  support  personnel,  and  leased  equipment;  and  costs  associated  with  the 
processing,  analysis,  comparison,  and  interpretation  of  airborne  results  generated  by  this 
demonstration. 
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Table  12.  Cost  elements  for  TEM-8  survey  demonstration  at  FKPBR  and  PBR-S12. 


Cost 

Category 

Sub  Category 

Details 

Quantity 

Battelle 
Cost1  ($) 

AMEC 
Costs  ($) 

Pre-survey 

(Start-up) 

Site  characterization 

Site  inspection 

4  days 

Mission  plan  preparation  & 
logistics 

18  days 

$31,434 

Calibration  site  preparation 

2  days 

$8555 

$5,822 

Mobilization 

Equipment/personnel 
transport  (includes  travel) 

3  days 

$9641 

Helicopter/personnel 
transport  (includes  travel) 

4  days 

$24,331 

Unpacking  and  system 
installation 

1  day 

$7073 

System  testing  &  calibration 

1  day 

$2796 

Pre-survey  subtotal 

$83,830 

$54,296 

Capital 

equipment 

System  use  rate  ($700/day) 

25  days 

$17,500 

Capital  subtotal 

$17,500 

Operating 

costs 

Data  acquisition 

Helicopter  time,  including 
pilot  and  engineer  labor 

18  days 
(74  hours 
airtime) 

$100,664 

Operator  labor 

14  days 

$8100 

Field  data  processing 

Geophysicist 

18  days 

$39,442 

Field  support/management 

Geophysicist/ manager 

18  days 

$24,256 

$17,544 

Maintenance 

Geo  so  ft  software 
maintenance 

$0 

Hotel,  air  fares,  and  per  diem 

Survey  team 

18  days 

$7267 

$4,687 

Fuel  truck 

Remote  refueling" 

4  days 

$800 

Airport  landing  fees  and  FBO 
fees 

18  days 

$1170 

Project  management 

4  days 

$6930 

Operating  cost  subtotal 

$188,629 

Post-survey 

Demobilization 

Disassembly  from 
helicopter,  packing,  and 
loading  for  transport: 

1  day 

$6391 

Equipment/personnel 
transport  (includes  travel) 

3  days 

$9821 

Helicopter/personnel 
transport  (includes  travel) 

3  days 

$18,364 

Additional  data  processing, 
analysis,  interpretation,  and 
reporting  (Oak  Ridge) 

$119,703 

Geophysical  investigation 

Mobilization,  validation, 
demobilization 

$106,719 

Reporting  (AMEC) 

$20,636 

Post-survey  subtotal 

$154,279 

$127,355 

Total  cost 

$444,238 

$241,914 

Total  costs  combined 

$686,152 

1  Includes  all  overhead  and  organization  burden,  fees,  and  associated  taxes 

2  Remote  refueling  was  required  only  at  the  PBR-S12  site. 

*A11  costs  reported  are  preliminary  estimated  costs  through  October  2009. 
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9.2  COST  DRIVERS 


The  major  cost  drivers  for  an  airborne  survey  are  the  cost  of  helicopter  services  and  the  data 
processing  and  analysis  associated  with  the  acquired  data.  In  terms  of  tasks,  these  constitute  the 
majority  of  the  field-related  costs  (i.e.,  mobilization,  data  acquisition,  and  demobilization  costs), 
which  represent  the  single  largest  cost  item  for  an  airborne  survey  project. 

As  mentioned,  helicopter  services  are  a  significant  component  of  the  costs  associated  with  the 
airborne  survey  project.  This  cost  element  is  included  in  the  mobilization,  data  acquisition,  and 
demobilization  tasks.  The  costs  include  helicopter  airtime,  fuel,  pilot,  aircraft  engineer 
(mechanic),  airport  landing  and  hanger  fees  (if  applicable),  and  per  diem  for  the  flight  crew. 
Depending  on  the  survey  location  (distance  from  home  base),  mobilization  and  demobilization 
costs  can  be  significant  when  compared  to  the  overall  data  acquisition  cost.  Additionally,  the 
type  of  survey,  weather  conditions,  length  of  survey  day,  terrain,  vegetation,  and  cultural  features 
will  greatly  influence  this  cost  element. 

Data  processing  and  analysis  functions  constitute  the  majority  of  the  remaining  costs  associated 
with  the  field-related  costs  for  a  survey.  As  with  helicopter  services,  mobilization  and 
demobilization  of  the  airborne  survey  equipment  and  the  geophysical  survey  team  is  also  a  major 
task  in  terms  of  cost.  This  is  typically  a  function  of  distance  from  the  home  base  or  previous 
survey  location  (i.e.,  if  shared  mobilization/demobilization  is  involved)  to  the  intended  survey 
project  site.  Peripheral  costs  associated  with  this  demonstration-validation  project,  such  as 
ground  truth  and  excavations,  are  not  part  of  the  cost  analysis  in  this  section  and  the  following 
section  (Sections  9.2  and  9.3). 

The  sensitivity  of  the  overall  cost  to  these  drivers  can  be  modeled  under  several  different 
scenarios.  Helicopter  time  on  site  is  a  factor  of  several  variables.  The  first  is  the  number  and 
dimensions  of  the  survey  blocks.  The  greatest  amount  of  non-survey  time  is  spent  in  turns  at  the 
end  of  each  line  in  preparation  and  alignment  for  the  next  line.  As  such,  fewer  and  longer  survey 
lines  are  more  efficient  than  numerous  shorter  ones.  Typically,  lines  longer  than  approximately 
3-5  km  do  not  gain  additional  efficiencies.  One  mitigating  factor  to  this  limit  is  a  pilot 
perfonnance  issue.  Longer  lines  typically  require  more  frequent  re-flights  since  it  is  more 
difficult  to  maintain  precision  flying  over  such  long  lines.  In  practice,  a  maximum  line  length  of 
5  km  is  recommended. 

As  discussed  above,  other  major  cost  drivers  are  mobilization,  data  processing,  and 
demobilization.  These  costs  are  a  function  of  project  size  and  transportation  distance. 
Processing  costs  and  data  delivery  times  typically  decrease  with  experience  at  multiple  sites. 

9.3  COST  BENEFIT 

This  section  compares  costs  of  three  different  survey  technologies.  These  include  man-portable, 
the  ground-based  MTADS  system,  and  the  TEM-8  airborne  electromagnetic  system. 
Operational  costs  for  the  TEM-8  system  are  equivalent  to  those  of  the  Battelle  VG-22  system 
because  of  their  similar  swath  width.  The  difference  in  swath  width  for  both  systems  results  in 
higher  cost  than  for  the  VG-16  vertical  magnetic  gradient  system,  which  has  a  12  m  swath  width. 
However,  as  noted  throughout  this  report,  the  TEM-8  system  was  designed  for  use  in  areas 
where  magnetometer  systems  are  inappropriate. 
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Based  on  several  sources  of  infonnation  regarding  the  deployment  of  ground-based  towed  array 
systems  on  a  UXO  contaminated  site,  five  scenarios  are  presented  for  the  purpose  of  comparing 
airborne  surveys  to  ground-based  surveys.  These  sources  of  infonnation  are  generally  informal 
and  include  discussions  both  with  industry  and  USAESCH  staff  experienced  in  the  application  of 
ground-based  towed  array  surveying  equipment  and  projects. 

Following  Harbaugh  et  al.  (2007),  we  assume  that  the  two  ground-based  technologies  might 
survey  only  2%  of  the  total  area  of  concern,  while  the  airborne  systems  would  survey  between 
2%  and  100%.  This  level  of  ground  surveying  has  been  used  in  ESTCP’s  WAA  Pilot  Program. 
We  also  include  higher  proportions  of  ground  surveying  for  comparison  purposes.  Harbaugh  et 
al.  (2007)  have  proposed  fixed  costs  of  $75,000  (mobilization,  demobilization,  reporting)  and 
acreage  costs  of  $500/acre  for  use  of  MTADS  at  two  sites.  We  assume  that  costs  for  a  towed 
EM61  array  would  be  roughly  equivalent  to  those  for  the  MTADS  towed  array.  Similarly, 
Harbaugh  et  al.  submit  fixed  costs  of  $45,000  plus  acreage  rates  of  $1540/acre  for  man-portable 
EM  surveys  at  these  sites. 

Comparisons  between  airborne,  vehicle,  and  man-portable  magnetometer  surveys  are 
summarized  in  Table  13.  These  scenarios  address  sites  of  1000  to  50,000  acres  of  geographic 
extent,  with  varying  rates  of  coverage  from  100%  to  2%.  TEM-8  airborne  costs  range  from  $136 
to  $291  per  acre  for  a  100%  coverage  survey  using  the  TEM-8  WAA  system.  These  costs 
include  a  nominal  $50,000  mobilization  cost  from  our  bases  of  operation  in  Tennessee  and 
Ontario,  Canada.  Airborne  costs  are  corroborated  by  recent  work  with  magnetometer  systems 
for  non-ESTCP  sponsors,  e.g.,  the  surveys  at  Kirtland  AFB,  Fort  McCoy,  Camp  Lejeune, 
Pinecastle  Range  Complex,  and  Fort  Ord. 

Man-portable  systems  generally  have  significantly  higher  acquisition  costs  than  airborne  systems 
(ranging  from  $500  to  $3000  per  acre,  depending  on  site  conditions);  are  extremely  time- 
consuming;  and  may  present  risks  to  personnel,  equipment,  and  the  environment.  Neither  the 
airborne  nor  the  ground-based  survey  costs  include  the  cost  of  excavation.  Comparison  of  the 
airborne  array  to  a  ground-based  towed  array  similar  to  MTADS  maybe  more  representative. 

The  extent  of  coverage  possible  with  an  airborne  system  renders  comparisons  to  handheld  man- 
portable  systems  somewhat  inappropriate.  It  is  readily  apparent  that  the  advantage  of  airborne 
surveys  over  ground-based  surveys  becomes  greater  as  the  area  of  concern  becomes  larger. 
These  figures  illustrate  that  for  EM  surveys,  man-portable  platforms  are  most  cost  effective  for 
sites  requiring  less  than  30  acres  of  actual  coverage;  vehicular  systems  are  most  effective  for 
30-400  acres;  and  airborne  systems  are  most  effective  for  sites  larger  than  400  acres. 


42 


Table  13.  Costs  per  acre  for  airborne,  ground  vehicle,  and  man-portable  survey 
platforms  for  varying  WAA  survey  densities. 

Shaded  cells  are  minimum  cost.  Man-portable  are  most  cost  effective  for  0-30  acres  actual 
coverage,  vehicular  systems  from  30-400  acres  and  airborne  over  400  acres.  All  costs  are  in 
thousands  of  dollars  and  include  fixed  mobilization  costs. 


TEM-8 


Acres 

100% 

50% 

25% 

10% 

2% 

1000 

$291 

$234 

$189 

$184 

$180 

2000 

$433 

$319 

$242 

$219 

$214 

5000 

$833 

$518 

$380 

$286 

$257 

20,000 

$2786 

$1456 

$852 

$541 

$365 

50,000 

$6835 

$3399 

$1893 

$995 

$511 

All  costs  are  in  thousands  of  dollars  and  include  fixed  mobilization  costs. 


Vehicle 


Acres 

100% 

50% 

25% 

10% 

2% 

1000 

$575 

$325 

$200 

$125 

$85 

2000 

$1075 

$575 

$325 

$175 

$95 

5000 

$2575 

$1325 

$700 

$325 

$125 

20,000 

$10,075 

$5075 

$2575 

$1075 

$275 

50,000 

$25,075 

$12,575 

$6325 

$2575 

$575 

All  costs  are  in  thousands  of  dollars  and  include  fixed  mobilization  costs. 


Man 


Acres 

100% 

50% 

25% 

10% 

2% 

1000 

$1585 

$815 

$430 

$199 

$76 

2000 

$3125 

$1585 

$815 

$353 

$107 

5000 

$7745 

$3895 

$1970 

$815 

$199 

20,000 

$30,845 

$15,445 

$7745 

$3125 

$661 

50,000 

$77,045 

$38,545 

$19,295 

$7745 

$1585 

All  costs  are  in  thousands  of  dollars  and  include  fixed  mobilization  costs. 


Number  of  Covered  Acres 


Acres 

100% 

50% 

25% 

10% 

2% 

1000 

1000 

500 

250 

100 

20  ! 

2000 

2000 

1000 

500 

200 

40 

5000 

5000 

2500 

1250 

500 

100 

20,000 

20,000 

10,000 

5000 

2000 

400 

50,000 

50,000 

25,000 

12,500 

5000 

1000 

Costs  for  MTADS  surveys  may  vary  from  those  estimated  in  Table  13.  The  following  was 
extracted  from  a  relevant  IDA  report  (Andrews  et  ah,  2001): 

For  this  demonstration,  the  MTADS  total  cost  was  $377,296.  If  the  excavation  costs  of 
$169,096  and  the  reporting  costs  of  $24,000  are  removed,  the  MTADS  costs  for  the 
deployment,  survey,  and  analysis  parts  of  this  demonstration  were  $184,200.  Note  that 
this  does  not  separate  out  the  costs  of  the  EMI  work.  The  MTADS  surveyed  a  total  of 
more  than  150  acres  for  a  cost  of  $1,222  per  acre. 
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According  to  the  IDA  report  conclusions,  “Cost  estimates  prepared  by  the  performers  indicate 
that  the  per  acre  cost  of  the  MTADS  is  about  2-3  times  higher  than  those  of  airborne  systems. 
These  figures  are  very  rough  estimates  and  may  not  accurately  reflect  the  cost  differences  seen  in 
operational  surveys.”  As  noted  earlier,  TEM-8  airborne  surveys  have  higher  cost  than  those 
referenced  in  Andrews  et  ah,  (2001)  due  to  their  narrower  swath  width. 

In  Table  13,  we  provided  costs  for  airborne  surveys  covering  between  2%  and  100%  of  the  area 
of  interest  with  ground-based  surveys  covering  2%  of  the  area  of  interest.  An  unresolved 
question  is  where  the  equivalency  would  lie  between  airborne  and  ground-based  technologies: 
Which  is  more  valuable,  a  10%  airborne  survey,  or  a  2%  ground-based  survey?  The  answer 
would  clearly  lie  in  the  ability  to  detect  the  ordnance  of  interest  at  the  site  for  both  systems  and 
the  uncertainty  about  ordnance  contamination  in  areas  that  are  not  surveyed.  The  greater 
sensitivity  of  ground-based  systems  must  be  balanced  against  the  probability  of  ordnance 
contamination  within  areas  that  are  not  surveyed.  The  choice  will  likely  vary  from  site  to  site. 
Ground-based  systems  have  more  cost  constraints  that  are  site-dependent  than  airborne  systems 
(e.g.,  unnavigable  terrain,  vegetation  that  must  be  cleared,  vibration-sensitive  ordnance),  and  this 
may  also  affect  the  selection  of  approaches. 

9.4  COST  CONCLUSIONS 

As  demonstrated  above,  comparing  costs  of  fundamentally  different  technology  approaches  is 
both  difficult  and  inconclusive.  The  previously  discussed  cost  comparison  provided  a  range  of 
answers  to  the  same  question,  namely,  what  are  the  costs  of  deploying  each  technology  over  the 
same  size  area  under  the  same  conditions? 

For  consideration  of  DoD-wide  application  of  the  airborne  technology,  a  number  of  factors  must 
be  considered  when  evaluating  the  appropriateness  of  the  airborne  technology  and  potential  for 
substantial  cost  savings.  While  initially  impressive,  it  is  not  possible  to  simply  apply  these  types 
of  cost  savings  across  the  entire  DoD  UXO  program.  Sites  must  be  of  sufficient  geographic 
extent  to  warrant  a  deployment  given  the  high  costs  associated  with  mobilization  and 
demobilization.  In  addition,  survey  objectives,  terrain,  geology,  vegetation,  and  cultural  artifacts 
must  also  be  considered  for  such  a  deployment.  Extremely  variable  terrain  and/or  the  presence 
of  tall  vegetation  can  greatly  limit  or  impede  the  use  of  the  airborne  technology  for  the  UXO 
objectives  of  interest.  Finally,  the  project  objective  must  be  consistent  with  the  detection  limits 
and  capabilities  of  the  airborne  system  to  make  such  a  deployment  feasible. 
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10.0  IMPLEMENTATION  ISSUES 


10.1  REGULATORY  ISSUES 

In  order  to  operate,  each  system  must  have  Federal  Aviation  Administration  approval 
(Supplemental  Type  Certificate  [STC]).  The  required  testing  and  evaluation  was  completed 
before  mobilization.  In  addition,  ground  crews  are  required  to  complete  the  40-hour  Hazardous 
Waste  Operations  and  Emergency  Response  (HAZWOPER)  course  and  to  maintain  their  annual 
8-hour  refreshers  for  operation  at  most  UXO  sites.  We  are  aware  of  no  additional  regulatory 
requirements  for  operation  at  the  FKPBR  site. 

10.2  END-USER  ISSUES 

The  primary  stakeholders  for  UXO  issues  at  the  FKPBR  site  have  not  been  specified. 
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